Theories of anterior cingulate cortex function : opportunity cost

The target article highlights the role of the anterior cingulate cortex (ACC) in conflict monitoring, but ACC function may be better understood in terms of the hierarchical organization of behavior. This proposal suggests that the ACC selects extended goal-directed actions according to their learned costs and benefits and executes those behaviors subject to depleting resources

Frontal midline theta and N200 amplitude reflect complementary information about expectancy and outcome evaluation

Feedback ERN (fERN) and frontal midline theta have both been proposed to index a dopamine-like reinforcement learning signal in anterior cingulate cortex (ACC). We investigated these proposals by comparing fERN amplitude and theta power with respect to their sensitivities to outcome valence and probability in a previously collected EEG dataset. Bayesian model comparison revealed a dissociation between the two measures, with fERN amplitude mainly sensitive to valence and theta power mainly sensitive to probability. Further, fERN amplitude was highly correlated with the portion of theta power that is consistent in phase across trials (i.e., evoked theta power). These results suggest that although both measures provide valuable information about cognitive function of frontal midline cortex, fERN amplitude is specifically sensitive to dopamine reinforcement learning signals whereas theta power reflects the ACC response to unexpected events

Reward feedback stimuli elicit high-beta EEG oscillations in human dorsolateral prefrontal cortex

Reward-related feedback stimuli have been observed to elicit a burst of power in the beta frequency range over frontal areas of the human scalp. Recent discussions have suggested possible neural sources for this activity but there is a paucity of empirical evidence on the question. Here we recorded EEG from participants while they navigated a virtual T-maze to find monetary rewards. Consistent with previous studies, we found that the reward feedback stimuli elicited an increase in beta power (20-30 Hz) over a right-frontal area of the scalp. Source analysis indicated that this signal was produced in the right dorsolateral prefrontal cortex (DLPFC). These findings align with previous observations of reward-related beta oscillations in the DLPFC in non-human primates. We speculate that increased power in the beta frequency range following reward receipt reflects the activation of task-related neural assemblies that encode the stimulus-response mapping in working memory

Feedback information and the reward positivity

The reward positivity is a component of the event-related brain potential (ERP) sensitive to neural mechanisms of reward processing. Multiple studies have demonstrated that reward positivity amplitude indices a reward prediction error signal that is fundamental to theories of reinforcement learning. However, whether this ERP component is also sensitive to richer forms of performance information important for supervised learning is less clear. To investigate this question, we recorded the electroencephalogram from participants engaged in a time estimation task in which the type of error information conveyed by feedback stimuli was systematically varied across conditions. Consistent with our predictions, we found that reward positivity amplitude decreased in relation to increasing information content of the feedback, and that reward positivity amplitude was unrelated to trial-to-trial behavioral adjustments in task performance. By contrast, a series of exploratory analyses revealed frontal-central and posterior ERP components immediately following the reward positivity that related to these processes. Taken in the context of the wider literature, these results suggest that the reward positivity is produced by a neural mechanism that motivates task performance, whereas the later ERP components apply the feedback information according to principles of supervised learning

Task-specific effects of reward on task switching

Although cognitive control and reinforcement learning have been researched extensively over the last few decades, only recently have studies investigated their interrelationship. An important unanswered question concerns how the control system decides what task to execute and how vigorously to carry out the task once selected. Based on a recent theory of control formulated according to principles of hierarchical reinforcement learning, we asked whether rewards can affect top-down control over task performance at the level of task representation. Participants were rewarded for correctly performing only one of two tasks in a standard task-switching experiment. Reaction times and error rates were lower for the reinforced task compared to the non-reinforced task. Moreover, the switch cost in error rates for the non-reinforced task was significantly larger compared to the reinforced task, especially for trials in which the imperative stimulus afforded different responses for the two tasks, resulting in a "non-paradoxical" asymmetric switch cost. These findings suggest that reinforcement at the task level resulted in greater application of top-down control rather than in stronger stimulus-response pathways for the rewarded task